Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Stackdriver Exporter and configs. #7

Merged
merged 5 commits into from
Jul 17, 2019

Conversation

songy23
Copy link
Member

@songy23 songy23 commented Jul 17, 2019

Fixes #6.

@songy23
Copy link
Member Author

songy23 commented Jul 17, 2019

/cc @dinooliva @rghetia

Copy link
Contributor

@pjanotti pjanotti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @songy23 not sure if you want to bring it as it is and latter improve it or change it before merging. In any case here are some small issues.

exporter/stackdriverexporter/config.go Outdated Show resolved Hide resolved
exporter/stackdriverexporter/stackdriver.go Outdated Show resolved Hide resolved
exporter/stackdriverexporter/stackdriver.go Outdated Show resolved Hide resolved
exporter/stackdriverexporter/stackdriver.go Outdated Show resolved Hide resolved
Copy link
Member Author

@songy23 songy23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @pjanotti for the review. This is mostly a copy-and-paste of https://github.com/census-instrumentation/opencensus-service/tree/master/exporter/stackdriverexporter with conforming to the new config and factory in otel-svc. You're right that there're certain things that are stale with the new configs.

exporter/stackdriverexporter/config.go Outdated Show resolved Hide resolved
exporter/stackdriverexporter/stackdriver.go Outdated Show resolved Hide resolved
exporter/stackdriverexporter/stackdriver.go Outdated Show resolved Hide resolved
exporter/stackdriverexporter/stackdriver.go Outdated Show resolved Hide resolved
exporter/stackdriverexporter/config_test.go Outdated Show resolved Hide resolved
exporter/stackdriverexporter/factory.go Show resolved Hide resolved
exporter/stackdriverexporter/factory.go Outdated Show resolved Hide resolved
exporter/stackdriverexporter/factory_test.go Outdated Show resolved Hide resolved
@pjanotti pjanotti merged commit 68b55bb into open-telemetry:master Jul 17, 2019
@songy23 songy23 deleted the stackdriver branch July 17, 2019 18:38
bogdandrutu pushed a commit that referenced this pull request May 12, 2022
* Add basic CI for unit testing
TylerHelmuth added a commit that referenced this pull request Sep 21, 2024
… Histo --> Histogram (#33824)

## Description

This PR adds a custom metric function to the transformprocessor to
convert exponential histograms to explicit histograms.

Link to tracking issue: Resolves #33827

**Function Name**
```
convert_exponential_histogram_to_explicit_histogram
```

**Arguments:**

- `distribution` (_upper, midpoint, uniform, random_)
- `ExplicitBoundaries: []float64`

**Usage example:**

```yaml
processors:
  transform:
    error_mode: propagate
    metric_statements:
    - context: metric
      statements:
        - convert_exponential_histogram_to_explicit_histogram("random", [10.0, 20.0, 30.0, 40.0, 50.0, 60.0, 70.0, 80.0, 90.0, 100.0]) 
```

**Converts:**

```
Resource SchemaURL: 
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope  
Metric #0
Descriptor:
     -> Name: response_time
     -> Description: 
     -> Unit: 
     -> DataType: ExponentialHistogram
     -> AggregationTemporality: Delta
ExponentialHistogramDataPoints #0
Data point attributes:
     -> metric_type: Str(timing)
StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2024-07-31 09:35:25.212037 +0000 UTC
Count: 44
Sum: 999.000000
Min: 40.000000
Max: 245.000000
Bucket (32.000000, 64.000000], Count: 10
Bucket (64.000000, 128.000000], Count: 22
Bucket (128.000000, 256.000000], Count: 12
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
```

**To:**

```
Resource SchemaURL: 
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope  
Metric #0
Descriptor:
     -> Name: response_time
     -> Description: 
     -> Unit: 
     -> DataType: Histogram
     -> AggregationTemporality: Delta
HistogramDataPoints #0
Data point attributes:
     -> metric_type: Str(timing)
StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2024-07-30 21:37:07.830902 +0000 UTC
Count: 44
Sum: 999.000000
Min: 40.000000
Max: 245.000000
ExplicitBounds #0: 10.000000
ExplicitBounds #1: 20.000000
ExplicitBounds #2: 30.000000
ExplicitBounds #3: 40.000000
ExplicitBounds #4: 50.000000
ExplicitBounds #5: 60.000000
ExplicitBounds #6: 70.000000
ExplicitBounds #7: 80.000000
ExplicitBounds #8: 90.000000
ExplicitBounds #9: 100.000000
Buckets #0, Count: 0
Buckets #1, Count: 0
Buckets #2, Count: 0
Buckets #3, Count: 2
Buckets #4, Count: 5
Buckets #5, Count: 0
Buckets #6, Count: 3
Buckets #7, Count: 7
Buckets #8, Count: 2
Buckets #9, Count: 4
Buckets #10, Count: 21
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
```

### Testing

- Several unit tests have been created. We have also tested by ingesting
and converting exponential histograms from the `statsdreceiver` as well
as directly via the `otlpreceiver` over grpc over several hours with a
large amount of data.

- We have clients that have been running this solution in production for
a number of weeks.

### Readme description:

### convert_exponential_hist_to_explicit_hist

`convert_exponential_hist_to_explicit_hist([ExplicitBounds])`

the `convert_exponential_hist_to_explicit_hist` function converts an
ExponentialHistogram to an Explicit (_normal_) Histogram.

`ExplicitBounds` is represents the list of bucket boundaries for the new
histogram. This argument is __required__ and __cannot be empty__.

__WARNING:__

The process of converting an ExponentialHistogram to an Explicit
Histogram is not perfect and may result in a loss of precision. It is
important to define an appropriate set of bucket boundaries to minimize
this loss. For example, selecting Boundaries that are too high or too
low may result histogram buckets that are too wide or too narrow,
respectively.

---------

Co-authored-by: Kent Quirk <kentquirk@gmail.com>
Co-authored-by: Tyler Helmuth <12352919+TylerHelmuth@users.noreply.github.com>
edma2 pushed a commit to edma2/opentelemetry-collector-contrib that referenced this pull request Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

exporter/stackdriver: move code and configs here.
3 participants